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ABSTRACT 


A deterministic simulation using a model of human dynamic 
orientation was written to optimize the parameters of the 
motion base control system for a six-degree-of-freedom flight 
Simulator. An experiment requiring pilots to rate different 
levels of motion fidelity during a basic flight task pro- 
vided a data base for validation of the simulation. Ratings 
between subjects for linear, rotational, and combined motion 
cues were inconsistent due, in part, to the subjects' lack 
Breeze e 1n MPHOME-I5 aircraft and proficiency in high 
performance aircraft. The coefficient of concordance among 
subjects for the three ratings were .4483, .4835, and .5914, 
respectivelv. Comparison of simulation results with ex- 
perimental data yielded positive correlations as high as 
.5138. Response of the simulation to changing wash-out 
filter parameters was investigated and found to be adaptable 


to experimental optimization methods. 
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I. BACKGROUND 


A. GENERAL 
The modern, complex flight simulator has assumed an 
expanded role in aviation research and development as well 
ESA military and Civilian training programs. In the face 
of the current energy crisis, manpower and budgetary cut- 
backs, and reduced flight operations, the Department of 
Defense (D.O.D.) has recognized the benefits of judiciously 
s q ПО simulator "flight" time for aircraft hours. 
Supplementing existing training programs with instruction 
їп flight crew trainers has long been an effective means of 
training safe, professional aviators. D.O.D. has establish- 
ed an objective of reducing flight operations 25% by the end 
of Fiscal Year (FY) 1981. The Chief of Naval Operations 
(CNO) has recently authorized the substitution of simulator 
time for ten percent of total annual flight time requirements 
(25 percent of the annual instrument flight time). Reference 
1 states the CNO's policy: 
"As additional simulators become available and more 
is learned on the 'transfer of learning' gained through 
the use of simulators, this program will be expanded." 
Increased emphasis on the use of synthetic training 
devices has accelerated the procurement and funding of 
special part-task trainers such as the Navy's Air Combat 
Maneuvering device and Universal Night Carrier Landing 
Ea tor. "Total D.O.D. FY// procurement funding for train- 


ing devices is 298.7 million dollars as compared to 88.5 
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million in FY74. Research and development funding during the 
same period has nearly doubled [Ref. 2]. The development and 
deployment of modern combat aircraft today is paralleled by 
the production of high-fidelity weapon system trainers. Ad- 
vanced weapon platforms such as the F-14, S-3A, P3-C, SH-2F, 
and the SH-3H have (or will) become operational concurrently 
with advanced, full task, motion base trainers. 

The following section reviews the current "State of the 
art" of the flight simulator as a tool in the training en- 
vironment, research and development community, and in the 
field of accident investigation. It references selected 
studies and events that demonstrate the devices' capabilities 
and effectiveness. 

К Жаат 

Understanding the benefits of flight crew trainers is 
essential to their cost effective use within comprehensive 
training programs. Efficient use of simulators reduces the 
expense of fuel, weapons, support equipment, and manhours. 

It also reduces the competition for hardware between training, 
operational, and maintenance departments. Alloted training 
hours can be devoted entirely to the task at hand, eliminating 
time for such activities as preflight, refueling, and clear- 
ance acceptance. As well as reducing accidents, it provides 

a safer environment for practice of hazardous or emergency 
flight operations. The advanced system incorporates fea- 
tures such as "freezing" the problem for real-time critique, 


replaying the maneuver allowing the student to objectively 
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review it while fresh in his mind, and computation of purely 
objective performance measures using computer based 
algorithms. (See Ref. 3 and 4 for examples.) 

Instruction in training devices provides better control 
of psychological factors such as stress and workload levels. 
Use of a select, limited number of experienced instructors 
enhances standardization of undergraduate and "fleet" pilots. 

The Naval Air Training Command, in 1972, began evaluation 
of a computer generated imagery system which complemented the 
existing TA-4 2F-90 flight simulator. The device provided 
visual scenarios such as aircraft carrier landings, a bomb- 
ing range complete with scoring system, and carrier catapult 
launches. The resultant success in the Familiarization Stage 
of the advanced student flight training syllabus was expressed 
in the following statements [Ref. 5]: 


"The major area of benefit has been the practice 
provided in VFR procedures and techniques. The addition 
of a carrier visual presentation has made student in- 
struction in CQ [carrier qualification] possible with 
much the same results as those experienced in FAM stage. 
The real benefit of the simulator however has proven to 
be in the weapons delivery phase of the syllabus. Bomb 
instructors have noticed a significant improvement in 
the students' ability to fly the pattern and develop the 
BINI OE bombing as a result of incorporating simu- 
lator flights into the weapons stage. ...After a short 
period of instruction, it was evident that students who 
had had several hops in the simulator were, to a signi- 
ficant degree, outperforming those who had not." 


"The real key to the value of the simulation is the 
hit-spotting program which provides the instructor with 
real time readouts of release altitude, release mach 
paar Eve angle, and hit position. ...The instructor 
Da Sales zimeröduce any of a- multitude of factors affect- 
ing mission completion including adverse weather, air- 
craft malfunctions, variable winds and many others 
affecting flight safety. ...The system has enabled the 
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instructors to provide the student aviator with a high 

degree of realism and a method of evaluation which, 

until now, has been unavailable even in the aircraft." 

2. Research and Development 
The use of cockpit simulators in the research, devel- 

opment, test, and evaluation phases of new aircraft allows 
the aeronautical engineer to develop flight systems and pro- 
Fr S writhout use of the prototype aircraft. Concurrent 
development of mockups provides for efficient aircrew station 
design and analysis from a system and human factors stand- 
emne. Design of displays and controls is simplified with 
construction of full-scale prototype crew stations.  Realis- 
tic, motion base simulators, driven by mathematical models 
derived from wind tunnel testing, allow test pilots to opti- 
mize aircraft performance capabilities and "handling qualities" 
Speer rc to the mission. 

Development of the U.S. Air Force/Boeing Advanced Medium 
STOL (short takeoff and landing) Transport was aided 
extensively by the use of Boeing fixed-base and NASA motion 
base flight simulators. The following advanced aircraft 
design concepts were evaluated on the devices before 
оорлоп into the prototypes [Ref. 6]. 

Ш engine bleed air for leading edge flap boundary 
layer control (BLC) 

Ш redundant mechanical and electrical flight controls 

Ш aerial delivery procedures 

Ш satisfactory handling qualities and criteria for 


engine-out STOL approaches 
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W "conventional" piloting techniques for STOL speed 
and thughtopathuacontsrol. 

An added benefit, recently used in the development of the 
F-14, is the capability to design, evaluate, and optimize 
specialized subsystems. The Automatic Carrier Landing Sys- 
tem (ACLS) of the F-14 "TOMCAT" was developed using a 
Sedegree of freedom (pitch, roll, and heave) NASA cockpit 
simulator [Ref. 7]. After "proper validation" of the flight 
dynamics, optimization of the parameters of the ACLS's three 
main components began. Use of simulated flights in lieu of 
aircraft flights yielded "significant increases and flexibil- 
ity in the number of combinations [of parameters] that can 
be examined for aircraft subsystems which have many parameters 
to specify." Other advantages of the program were soon 
realized: 


"1. Allowed seven iterations of SPN-42 [shipboard 
guidance equipment] parameter values to be examined in 
two to three days instead of the normal two to three 
weeks and several flights. 

o Provided qualitative predictions of pilot com- 
ments on closed loop ACLS control characteristics and 
quantitative prediction of performance statistics, pilot 
АОВ (ACLS Quality Rating), and to a lesser extent, open 
and closed loop frequency responses. 

3. Saved $120,000 and $150,000 and approximately 
six months in the SPN-42 optimization phase of ACLS 
development." 


"The impact of this new T&E capability is indicated 
DN ncluston of specific periods of simulator util- 
ization in the ACLS development plan for the S-3A airplane. 
Soil uae zation will likely be specified for the F-18 
airplane. In addition, the same NASA-Ames Research Center 
R&D simulator used for the F-14A has been reprogrammed to 
attack long standing ACLS problems of the A-7E and A-6 
eercraft.” 
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3. Accident Investigation 


Cockpit simulators have proven to be an indispensible 
mee! for reconstruction of pre-accident histories. The pres- 
ent capability to use data from inflight recorders to derive 
Mee profiles, environmental conditions, and signals with 
which to "drive" simulators is impressive. Current digital 
recorders can record up to 64 flight parameters per second. 
However considerable re-formatting of the data is then re- 
quired in order that simulators can use the data directly. 

A current NASA program is in pursuit of the capability to 
recreate pre-accident flight conditions using information 
from flight recorders, voice recorder data, radar tracks, and 
meteorological data. 

The investigation into the crash of an Eastern Air- 
Tines B-727 in June, 1974, by the National Transportation 
Safety Board included evaluation of the wind shear and down- 
draft activity encountered by the aircraft [Ref. 8]. Four- 
teen experienced pilots flew a total of 54 approaches in a 
fixed-base B-727 cockpit simulator programmed with four 
different hypothesized wind models. Even with an a priori 
knowledge of the type and severity of the storm activity, 

IG CS crashed, while only five flights resulted in 
placement of the aircraft near the runway threshold. Subse- 
quently constructed flight profiles (airspeed and altitude 
traces) closely resembled those from the B-727's flight 
recorder, validatıng, to some extent, the wind shear models. 


At the time of the crash, the flight crew was thought to be 
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attempting visual contact with the runway environment. Seven 
of ten pilots who commented following the tests felt that 
their efforts to "go visual" at decision height delayed their 
recognition of the sink rate produced by the wind shear and 
downdraft conditions. 
A committee, chaired by the Director of Aviation 
Safety Programs, Naval Postgraduate School, was formed in 
1975 to investigate present use and potential of flight sim- 
ulators in aircraft accident prevention and investigation. 
Representatives from government, NASA, and the airline indus- 
try recommended further research into the use of existing 
inertial navigation equipment to record available flight 
accelerations in a form adaptable to simulators [Ref. 9]. 
The committee offered these recommendations: 
oaasi protected digital flight recorders should 

рема еа in high valued aircraft (with inertial 

navigators). There should be a unified effort to 

develop and implement standardized data collection and 

processing procedures which would facilitate recreation 

ers cendent/ineldent conditions in flight simulators. 


Voice recorders, especially on multi-crew aircraft, 
should also be considered." 


Se NATURE OF THE PROBLEM 
J s Ootc Simulator Feedback Loop 
while accomplishing a certain task in a flight sim- 
Or EM pilot responds tO various visual, kinesthetic, 
audio, and vestibular feedback cues (see Fig. 1). Feedback 
may bewclassified as either "intrinsic" (e.g., control "feel" 
or amplitude of movement necessary in the operation of a 


Control) or “extrinsic” (the consequences or results of 
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control operation). The problem encountered by a designer is 
one of cost-effectiveness, i.e., which cues are necessary and 
what degree of fidelity is required of each cue to ensure a 
positive transfer of training at the particular stage of 
learning for the given task. 

The requirement for motion fidelity in a flight sim- 
ulator has been an ongoing question for years. It cannot be 
debated that motion cues simulating airframe buffeting as 
well as in response to flight control inputs adds greatly to 
psychological fidelity, especially during early stages of 
training. Many research studies have compared fixed-base 
and motion base simulators measuring pilot performance, 
learning rates, and subjective pilot responses. Where 
motion cues play an important role in the pilot's feedback 
loop, the addition of motion cues creates a significant 
improvement in performance. For example, during instrument 
approaches, the inherent lag in cockpit instrumentation 
forces the pilot to rely on the vestibular sense to a higher 
degree. (A possible argument for fixed-base instrument 
training, where the pilot is forced to depend on artificial 
cockpit references). During visual carrier landings, the 
first sensation of rapid glide path deviation is typically 
through the vestibular and kinesthetic sensors, quickly 
confirmed by movement of the visual landing aid. Another 
scenario in which motion cues are important to the pilot 15 
low-level, high speed terrain following flights. Where motion 
tends to degrade pilot performance, the addition of these cues 


allows a quicker transfer of learning to the aircraft. 
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2. Physiology of Motion Detection 


Man perceives the sensation of motion throuch three 
main sensors. The eyes view the changing world around him. 
The kinesthetic sensors within his body respond to the rate 
of change of movement of his body and its members. The semi- 
circular canals and otolith organs are the main components 
of the highly specialized vestibular system of the inner 
ear. The inner ear canals react to changes in rotational 
uehecitv, i.e., angular acceleration. The utricles and 
saccules of the otolith system respond to linear accelerations. 

The present intent is not to analyze in depth the 
physiology of the vestibular system, but to review the mo- 
tion to which the organs respond. (A more comprehensive 
review of vestibular physiology and its function is available 
im Ref. 10) 

The fluid-filled semicircular canals form a roughly 
orthogonal set of sensors in each inner ear. The endolymph 
fluid lags behind the movement of the canal walls when the 
head undergoes rotation. This relative motion tends to dis- 
place the cupula which obstructs an expanded portion of the 
canal called the ampulla. Deflection of the cupula excites 
sensory hair cells at its base causing an increase in the 
firing rate of the afferent nerve fibers. Orientation of 
the canals is such that those on the left side of the head 
are coplanar with those on the right side of the head, 
sensing motion about the same axes, but in the opposite 


direction, thereby acting as a single unit. It can be 
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assumed that the central nervous system responds to the 
difference of their respective afferent responses. 

The otolith system of the inner ear is comprised of 
Bo uEericles and two saccules. Each is a mass of calcium 
carbonate crystals suspended by sensory hair cells. Linear 
accelerations tend to displace the mass exciting the af- 
Benentonerve endings at the base of the hair cells. The 
utricle organs are oriented so as to be sensitive to the 
horizontal component of specific force. The saccule organs, 
approximately perpendicular to the utricles, sense vertical 
shear forces and are roughly one-half as sensitive. 

3. Vestibular Modelling Research 

The need for development of a model of the human 
vestibular system can be illustrated best when one considers 
the advances made in modern aerospace technology. The 
high G environment of combat aircraft and space launch 
vehicles, the prolonged weightlessness of spacecrafts, and 
the motion experienced in large ships and tall buildings are 
products of twentieth century technology requiring the 
understanding of man's capacity to function in these 
environments. 

Mies State of the art" of current modelling capabil- 
ities is best illustrated in a PhD thesis by Ormsby [Ref. 11]. 
At the MIT Man-Vehicle Laboratory, under a NASA grant, Dr. 
Ormsby developed a mathematical model based on the known 
physiology of the human vestibular apparatus. The signal in 


nosse model is capable of predicting near threshold and 
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supra-threshold perceived motion and attitude in the pres- 
ence of random noise and spontaneous firing of the afferent 
sensors. It was then refined to be consistent with current 
neurophysical and psychophysical data. Included is a model 
Si ehe central processor (the brain) and its interpretation 
of afferent dynamics with an à priori knowledge of the mo- 
m E Cimilus. Since it was difficult to obtain data 
resulting only from vestibular cues, the model is said to 
also reflect responses resulting from tactile and kinesthetic 
motion cues. 

Separate models of the semicircular canals and the 
otolith organs are combined to yield the following estimates 
in the coordinate system of the head: 

B perceived linear acceleration 

Ш perceived rotational velocity 


B perceived attitude (a unit vector in the perceived 
"dem direction). 


The model has been successful in predicting illusions 
such as the dynamic elevator illusion in the absence of vi- 
sual inputs. It also has produced results in close agreement 
with data describing perceived lateral tilt angle as a func- 
tion of actual tilt angle in "1G" and "2G" environments. The 
pitch and roll sensations during a catapult launch were 
investigated and used to further refine the model. 

It must be emphasized that the version of the model, 
as used in this thesis, is strictly a vestibular model with- 
out visual, tactile, or kinesthetic inputs. The integration 


of vestibular and visual cues is the subject of an ongoing 
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ES I ol program at the MIT Man-Vehicle Laboratory 
Mets, 12 and 13]. 
4. Components of a Motion Base Control System 

The human vestibular system is sensitive, not to 
motion, but to the higher derivatives of motion such as 
linear and angular acceleration. Thus only the "rate of 
change" of motion need be simulated. The simulation prob- 
lem, then, is how to optimally simulate acceleration cues 
given fixed operating limits. (Hardware has inherent size, 
velocity, and inertial limitations, while computers have 
speed and accuracy limits.) A computer model of the air- 
craft produces acceleration vectors in response to control 
inputs and external factors such as wind and turbulence 
which are converted to simulator drive commands. Linear 
accelerations are simulated by tilting the cockpit cab 
(such as backward on application of thrust) at a rate be- 
low human rotational thresholds. Thus lateral and longitu- 
dinal acceleration cues can be simulated, but only the 
higher frequency vertical cues (such as turbulence and 
rapid pullups) can be simulated. After the cue has been 
transmitted, the cab is gradually returned tc its neutral 
position. This tendency, known as "wash-out", keeps the 
cab near its center position to maximize its operating 
envelope for subsequent cues. 

This section describes the main components of motion 
base control systems. The vast majority of motion base 


simulators have either "synergistic" platforms supported and 
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driven by linear hydraulic actuators or electro-mechanical 
drive systems with independent degrees of freedom. Digital 
computer programs for both system types are comprised of 
three main components: 

B aircraft models which produce the particular 
flight dynamics and control feedback ("feel") 
characteristics 

Ш "wash-out" filters which remove low frequency 
accelerations which tend to generate large 
amplitude movements 

B "drive" programs which transform the aircraft 
motion to simulator motion commands. 

This sequential process is shown in Figure l. The motion 

of the aircraft about its six degrees of freedom is used to 
drive cockpit instrumentation, update navigational equipment, 
and drive external visual displays. The drive program 
consists of algorithms which compute: 

Ш residual tilt which uses gravitational forces 
to simulate linear accelerations 

Ш lead compensation for "quickening" the position 
commands to compensate for inertial lag in the 
machine's dynamics 

B cab-to-drive mechanism transformations 

NEU SUNDSSICtUIOD, velocity, and acceleration 
limiting networks. 

Although many additional safety features and utility func- 


tions are present in most programs, these are the primary 
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motion generating components. (See Refs. 14 and 15 for more 
Is I p (D; theoretical descriptions of control systems.) 
oe Simulator Fidelity 

Studies which evaluate motion control systems employ 
sophisticated means of calculating performance measures and 
compiling subjective pilot responses. Numerous studies have 
evaluated the handling quality differences between fixed 
and motion base simulators in an effort to understand the 
meee Gor and the contribution of motion cues to the pilot. 
(See Refs. 15 and 16 for examples.) Improving stability 
characteristics for new aircraft or simulators often in- 
volves the analysis of pilot performance while system param- 
eters are systematically changed. Verbal responses are 
analyzed to discern which aspects of the motion cues are 
deficient. Design of an aircraft control system typically 
entails a series of compromises and tradeoffs yielding 
handling qualities seldom optimum for any specific operating 
regime. Therefore, optimum performance does not reflect the 
fidelity of a motion system. Approximate duplication of 
such fundemental maneuvers as coordinated turn entries has 
been accomplished by matching aircraft and simulator accel- 
erations about all six degrees of freedom. Since a precise 
duplication is unattainable due to the simulator's physical 
constraints, only the human operator can properly evaluate 
the differences. 

Optimization of the control system for a new simu- 


lator is typically accomplished through numerous flights by 
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an experienced test pilot while engineers vary stability 
parameters. Many unpublished fidelity programs involve 
hundreds of manhours and expensive computer time in search 
of optimal control parameters. Often pilots need refresher 
flights in the aircraft to regain familiarity with its 
stability characteristics. One such study [Ref. 22] esti- 
mated that "eight hours of simulator time is sufficient to 
destroy a pilot's familiarity with the airplane to the point 
that he can no longer make valid qualitative judgements on 


flying qualities during closed loop tests." 


C. PRESENT PURPOSE 

Present D.O0.D. emphasis is on new, higher fidelity 
flight simulators. A second avenue of approach is the 
improvement of present training devices having less than 
optimal flying qualities. Lack of fidelity has reduced the 
purpose of many devices to mere procedural trainers, while 
the motion and visual systems go unused. With new technology 
visual display systems, the shortcomings of the handling 
qualities and performance characteristics are magnified. 
As simulators assume a greater role in flight training and 
proficiency programs teaching advanced flying skills, the 
need for realistic duplication of the aircraft is apparent. 

The Naval Air Test Center (NATC) has been conducting a 
fidelity improvement program for the Navy's inventory of 
Operational Flight Trainers and Weapon System Trainers 
[Re£. 22]. A team of fleet experienced pilots, flight test 


engineers, simulator and computer experts have been 
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successful im identifying performance, stability, and con- 
Baer parameters contributing to poor fidelity in all the 
subsystems. Although some hardware changes have been made, 
the majority of fidelity improvements have been accomplished 
by software reprogramming. 

"Aircraft manufacturers generally rely on wind 
tunnel data as a data base. Flight test data are not 
available because the first training device is usually 
delivered to the Fleet at about the same time that the 
first airplane comes off the production line. Since 
wind tunnel data represent at best only an estimate of 
airplane flight characteristics, the use of these data 
as a data base for a simulator results in poor flyina 
qualities simulation. In addition, there has been no 
organized effort within the Navy or by any manufacturer 


to reprogram a simulator once flight test data are 
avallable." 


"Recent advances in the practical applications of 
control and estimation theory have made it possible to 
extract stability derivatives and other parameters 
from flight test data to a degree and with accuracy 
not possible in the past." 

The degree of realism required in equipment, visual dis- 
pu ы ап motion systems is a function of the level of 
training being performed and the amount of learning transfer 
desired. For a specific training device, whatever cost- 
effective measures (up to the point of diminishing returns) 
increase its fidelity would be welcomed, thus improving the 
transfer of learning. 

The goal of this thesis is the development of a technique 
which would allow the simulator programmer (such as those of 
NATC's fidelity improvement program) to optimize the archi- 
tecture and choice of parameters for simulator software. 


Work on an ongoing research program by Dr. L. R. Young at 


the NASA Langley Research Center and the MIT Man-Vehicle 
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Laboratory on the integration of visual and motion cues 
provided many original ideas for this work. An experiment 
was performed at the NASA Ames Research Center in the Flight 
Simulation Laboratory on a six-degree-of-freedom research 
simulator. The experiment was designed to test the hypothe- 
sis that there was no difference between simulator motion 
cues presented to the pilots. Five pilots performed a total 
of 161 simulator "runs", providing subjective ratings of 
different degrees of fidelity. 

Using Ormsby's model of human dynamic orientation, a 
computer program was written to simulate the pilot's subjec- 
tive evaluation of fidelity in objective terms. Validation 
of the simulation was performed using the experimental re- 
sults as a data base. Another aim of this report 1s to 


confirm Dr. Ormsby's model to whatever extent possible. 
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II. METHOD 


A. EXPERIMENT 
l Desi 

The pilot-subjects were asked to provide responses 
to different levels of motion fidelity. Fifteen sets of 
"wash-out" filter parameters were selected in an effort to 
produce discernably different motion cues during a flight 
task. For each "flight", subjects were asked to rate the 
rotational and linear acceleration cues relative to a 
rating scale and to provide a combined fidelity score. 
Thus, the dependent variables for the experiment were the 
three subjective ratings of the motion cues. The indepen- 
dent variables in the analysis were the motion cues and 
parameter sets. 

The experimental results were analyzed according to 
a three-way factorial analysis of variance. Two fixed fac- 
tors were motion (three levels) and parameter sets (15 
levels). The third was subjects, a random factor (five 
levels). A conceptual model of the experimental design is 


shown in Figure 2. 


28 





1 
22 
: P15 

SUBJECTS S3 

PARAMETER 

S — == 

4 SETS 

55 





MOTION CUES 


Figure 2: Model of Experimental Design 
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Since the simulator was programmed as an F-15 (a 
combat fighter aircraft), subjects for the study were chosen 
for their experience as pilots in high performance aircraft. 
Although the subjects obtained for the experiment were ex- 
perienced in military combat aircraft, only one was relative- 
ly current. The other four subjects were military trained 
airline pilots or flight engineers. Their pilot time exper- 
ience is shown in Table I. Each was a volunteer and was 


compensated for his participation. 
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SUBJECT HELO SINGLE-ENGINE MULTI-ENGINE HIGH PERFORMANCE JET 


A a TH AAA 
E A Ban SS pt | 


L = 550 2400 3050 
2 = 180 860 130 
3 600 160 35:0 250 
2 E 30 - 800 
5 = 1600 7500 2150 


Table I: Subject Experience (pilot hours) 


23 Stimuli and Apparatus 
a. Simulator 

The experiment was performed at the NASA Ames 
Research Center, in the Flight Simulation Laboratory. The 
simulator used in the experiment was the 5.01 All-Axis 
Motion Generator, manufactured by the Northrop Corporation. 
The six-degree-of-freedom research simulator has independent 
drive mechanisms with a single-seat cockpit in the center of 
its operating envelope. The machine's linear position limits 
are +9 feet; the rotational limits are +45 degrees from its 
neutral position. The simulator was controlled by an EAI 
8400 computer. The computer programming included "soft" 
position, velocity, and acceleration limiting networks which 
prevent the simulator from reaching the "hard" mechanical 
limits. A more in-depth study of the machine's operating 
characteristics and mathematical development of its "drive 
logic" is contained in Reference 15. 

As with most simulators, the cockpit was en- 


closed preventing visual reference to outside of the cab. 
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The cockpit configuration was that of a typical single-seat 
aircraft, controlled by a stick, rudder pedals, and power 
lever. Since visual cues from external displays and arti- 
poeta attitude instrumentation tend to interact with 
vestibular cues in a way not fully understood, use of these 
displays was minimized. (The simulator was equipped with a 
terrain modelboard visual system, but the projection equip- 
ment was off for the duration of the experiment.) The main 
instrument panel was organized in a conventional "T" arrange- 
ment as shown in Figure 3. Although the basic instrument 
array included an artificial horizon (attitude gyro), it 

was felt that its absence would impose an unusually high 
workload level on the subject and tend to lengthen the learn- 
Boe Process inherent in the task. Also, the more familiar 
the cockpit environment was to the subject, the quicker 
would be his adaptation to the task, allowing maximum 
mr I On to the motion cues. 

An intercom system, with a "hot" microphone for 
the pilot, allowed two-way communication between the subject, 
System operators, and experimenter. Verbal responses by the 
pilots were recorded on a cassette tape recorder. The ex- 
perimenter monitored the progress of the task on a strip 
chart recorder, displaying altitude, airspeed, vertical 


Speed, and heading. 
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Figure 3: Main Instrument Panel 


The "wash-out" filter, which interfaces the 
mathematical aircraft model with the "drive" program, 
is composed of six separate digital filters, one for each 


degree of freedom or "channel". Each second order, high 
pass filter is described by a linear gain, a natural 
frequency, and a damping ratio with a Laplace transform 


equation of the form: 
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asura bn i reguency 
Througn consultation with NASA personnel, 15 
sets of parameters (18 parameters each) were chosen to 
produce noticeably different levels of realism. Values of 
the filter parameters are presented in Appendix B. The 
fidelity associated with each set could not be vredeter- 
mined, except that the range over which each parameter 
varied was thought to be sufficient to produce noticeable 
differences in motion cues. 

b. Task 


Initially, the experiment was designed to ana- 


lyze two different tasks in order to examine any significant 


difference in subject responses. Due to time constraints 


and equipment malfunctions, the second task, an ILS (instru- 


ment landing system) approach to a waveoff, was eliminated 


from ene study. 


Design of the task reflected many considerations. 


It was recognized that the task should be: 


Y similar to those used in RDT&E and training programs 


so that the maximum extrapolation of results could be 


made 


B relatively familiar to the subjects and well within 
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their capabilities to lessen the time spent mastering 
ie 

Ш well within the capability of the simulator to mini- 
mize machine error and breakdowns 

B long enough to expose the subject to a full variety 
of motion cues and allow him ample time to subjectively 
evaluate them. 

A three and one-half minute "basic airwork" task 
was chosen to simulate the low frequency flight maneuvers 
common to the terminal phase of flight operations (climb-out 
acer takeoff, descent prior to approach and landing, etc.). 
Subjects were provided enough information to rapidly learn 
the task and to execute it consistently from trial to trial. 
As shown in Figure 4, two versions of the flight pattern 
were devised. The direction of the initial turn was random- 
ized. During the second replication of each parameter set, 


the turn was made in the opposite direction. 


C. Rating Scales 

Subjects were asked to subjectively judge the 
fidelity of the motion cues encountered during the task us- 
ing two rating scales. The pilots scored separately the 
linear (or translational) and the rotational motion cues on 
a graphic eight-point scale. The cues were also rated in 
comparison to a "standard" motion (presented using the 
established set of wash-out parameters employed on the simu- 
lator). Thus, the subjects rated the motion fidelity on 


both an absolute and comparative basis. 


34 





OE 


ПЕШИ = 
әзеч QUIETO 
:ӘЗОМ 


HSINIA 

E AA 
uain 

ə}ey pIepueas 
:2720N 





илхәззеа увер IUBTTA  :v Sanbta 








(spuooəs) 
&— oE —> Sh -> 09 —  —=— Sk; St 0€ —A : ƏUIT L 
“€ 00€ ST: ¿€ ST: I 0€ * 00: l 
° ° ° 2 : :bur33eS WdH 
i 504 SOL | SVL | SE | „u l surbug "xoaddy 
MENE 
000€ 000€ 
005? UML po Triru —— 
O UnL A mg 
SWIM 077 
:p99dsıTy 
LUWLS 
:butpeeH 





35 





әтео$ Битеу Yusuoduosg 


eTqeyzotpeidun ^отуеллә ‘wstpesaa 
jo asuas Aue б$хуоет ’DTIETTOIJUODUN 


9TIST[Pe31UN Аләл 
^'o[qerro4dquooun Атлеәц 'Sərouərorjəp 


а-а... 
ѕ5016 Ѕполәшпи “ərqeadəoocun 


отз5ттеәлип 
*S9TIST1939e1ey9 3uesea[dun Auew 
’SOTIUSTITFOP ло{юш 'snodoumnw 


pepeeu jueueAoddur uonu 


‘әтаецотзоә[цо ÁAadoA 'soroueroIjop 
ло(еш Диеш/лоитш ^'snoaioumw 


ot3sr[?o3 Á[o3e3dopou 'popoou 
jueusAoiddur 'sorouorotJjop 
atqeuorzoal(qo ‘zoutw ÁAuew 


pəirsəp quəuəAoadur 
: ا‎ 
'S9rOU9STOIJop AOUTW May 


рәлтпһәл 
aqauəuəAoaiadur ou “ətqeəorqou snf 
^queorgtubrsutr ло qaubrqs ‘may 


IOTISTL[EeD9A A[əuoəi хә 
“'ƏtqtuaƏoSsIp DUON 


SALONATOTIAG NOI LOW 


2G әлобтя 


9TIP119 
‘aTqeqsun ^әтдеттолуцооцп 'əatsuodsəiun 


əlqrssodur Burioeag “d3əAnəueu oq pəarnbəa3 
uorqesuəcduio5 pue uoyquəqqe jo IƏAƏI ubru 

A [au913x32 “əlqeqasun uəqjJo 'puodso3 o3 yornb 
10 ysrbbn]s Атәшәлухә ‘atqeydaooeun 


poatnbar uorquəqqe jo [9A91 o3eurpiüour 
'e[tqrssodur Apaeou burxoegd3 S11 ‘Toaquos oy 
1["2I3JIP '[993 ,3ubrq, 10 ,Áneay, 'esuodsoi 
xXornb Aq[319AO ло Mots Arana ’AA0oydejstyesu 


310339 Je3ab yyıIm ÁAq[uo burx2e13 SI 
pearnboa s3uouoAoJdJdur o(eu “1004 





obae[ oo3 10 [Tews 00 UOT OU 
Толучоо рәлтиБәл ^тлоўўә әтделәрт5сиозэ 
цатм burxa9e13 STI '*pəairnbəi uorquə3qe 


їїў “Əsuodsəi usID5Dburs 10 yoınb ^лтед 


pəirsəp 

J3uəuəAo3adur ^'3j303jo9 әзеләрош YJIM 

buryoea3aq ^рәлүпБәл [әләт uorquo33e 
Əyeaəpou “sarsuodsa1 “,IƏƏJ, POOD 


pəpəəu 

yuəuəAoidur ou 'oseo цатм Hhuryoesaq 
SII 'pəirnbəi uorquəl3e [ewaou 
“'ƏArsuodsəi ',199J, poob X19A 


SrasrIeə3 
Atybrı “sea Vy. oa uorquəqqe 
[?*uazou soatrnboa 'oArsuodsoa 


Ád1OA 'u300us ^,][99J3, 3It9o[T99OX4g 





= — — — — 


SLOANE IOULNOD OL HSNOISAU 





TIVOS ONIJLVNH LNANOdWOO 


IWNOTGLW.LOU 


t) 





AVANT 


36 











o[eog BUTIPY DSATIRAPAUO) :9 әлпбтд 


1e3 Аа рәлләдәла st ,p1epuez3s, 
tzorzəgur ATUBTY ^әвлом цопи 


рәхләјәла st „pIepue]3Ss 
9SIOM 


pa11a3exd ATYAUBTTS ST „PIepue3s, 
AOTASJUT Атаеәотзоц 23snf 


„paıepue3s, ey of ATayhHt ts ÁATuo poeaaogead 


[r 
19339q ¿ATqeəorqou qasnp 


„рлериез5„ әчз оз рәлләјәла 


et 
193399 


хеј Аа „paepue3s, oy o3 peaudegeud 


aotıadns ATyßrty ‘әтделтѕәр әлош uonw М 


y UOTION PA1PPUBIS, ou3 o4 uosraeduoo ur SOTI3STI2J93OPJ2PuO UOT3OU 9u3 әзе 


J'IVOS SNILVH SHAILLVNHVdWOO 


37 





4. Procedure 

After the subjects read the instructions (see Appen- 
dix A), further questions were answered. The intent and 
goals of the experiment were discussed freely with the sub- 
Jects. They were allowed to become familiar with the simu- 
lator cab during one practice run before the trial runs 
began. Each was given a pilot kneeboard containing a dia- 
gram of the task, the two rating scales, and a list of topics 
from which relevent comments were volunteered following each 
mn. 

The sequence of wash-out parameter sets presented to 
the subjects was randomized; a different sequence was pre- 
sented to each subject. The "standard" motion was presented 
once at the beginning of each session (twice at the start of 
a subject's first session) and every fourth run thereafter. 


A typical session of 12 simulator runs is shown in Figure 7. 


0 14 0 5 
where: Py = "Standard" parameter set 
HUE Typical Sequence of Parameter Sets 


No reference was made to the parameter set number or their 
values during the trial runs. 

At the end of each task, subjects rated the motion 
cues and provided subjective comments relevent to the spe- 


cific run via the intercom. The parameter set was changed 
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and the simulator was reinitialized for the next run. After 
each session, subjects were allowed to review their responses 
and discuss the session (comments were not recorded). 

Factors which were uncontrollable were the number of 
sessions and runs per subject. Due to equipment malfunctions 
and scheduling constraints the sessions and runs varied 


widely as shown in Table II. 


SUBJECT NO. SESSIONS NO RUNS 
L 3 24 
2 3 37 
3 å 42 
+ 2 26 
5 2 21 


*Includes "Standard Motion" Runs 


Table II: Sessions and Runs by Subjects 


Be. SIMULATION 
1. Model Assumptions 

The subjective process by which a pilot compares the 
motion of a simulator with that of an aircraft is, obviously, 
a complex psychological one. If a model of this human com- 
parative method were constructed, it could predict a pilok's 
response to the fidelity of a simulator's motion character- 
istics. The structure of the computer simulation described 


in this section is based on the assumption (as proposed by 
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Young, Ref. 13) that the pilot's evaluation is a function of 
the difference between: 
Ш the simulator motion cues as perceived by the pilot 
and 
Ш the perceived motion of the aircraft as experienced 
By che pilot. 

Assuming that Ormsby's vestibular model of human dy- 
namic orientation provides a reliable estimate of motion 
perception, 1t was then a matter of deriving motion histories 
Simew@ewaircratt and the simulator performing “identical” 
maneuvers. 

2. Design 

During the experiment described in Section A, the 
output of the computerized aircraft model (see Fig. 1) was 
recorded on magnetic tape during each run. The data con- 
Sisted of accelerations for each degree of freedom recorded 
during each cycle of the discrete programming. The cycle 
time used in the experiment was 50 milliseconds, chosen so as 
to De compatible with the input sampling rate of the dynamic 
orientation program. (Sufficient data was also recorded to 
document the specific pilot and run number.) 

The flight acceleration data was used as an input to 
the S.01 simulator's drive program, a copy of which is used 
in this simulation. The simulation is a deterministic model 
with no random variables. Figure 8 is a block diagram of the 
simulation. Since the dynamic response of the motion platform 


contributes to the motion cues presented to the pilot, but is 
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not a factor in this simulation, the lead compensation net- 
work in the program is bypassed. The result is the closest 
approximation available to the actual motion of the cab. 
The outputs from this program are linear acceleration com- 
mands and rotational position commands of the platform's 
drive mechanism. A transformation then converts these sig- 
Habs back to vectors describing the cockpit motion. The 
rotational position commands are then differentiated to 
produce rotational velocity vectors. 

Inputs to the human dynamic orientation program are 
discrete time histories of specific force vectors (in units 
of "G's") and angular velocity vectors (radians/sec.) in the 
coordinate system of the pilot's head. It is sampled at dis- 
crete intervals of 100 milliseconds; thus the simulator and 
aircraft motion vectors are sampled at every other time 
frame. Although this represents a loss of input information, 
considerable recomputation of many model parameters would 
have been necessary to alter the input or output rates. The 
output of Ormsby's model consists of vectors in the coordi- 
nate system of the subject describing his perception of 
rotational velocity, linear acceleration, and attitude (per- 
ceived vertical or "down" direction). The model's output 
rate 1S one sample per second. 

Motion vectors of the aircraft are computed from the 
ile data after the rotational velocity is formed by inte- 
grating the rotational acceleration. These vectors are then 


used as inputs to the dynamic orientation program to produce 


42 





perceived aircraft motion. Notation used in the block 


diagram is defined as follows: 


Tor Odell linear aná rotational acceleration 
vectors, respectively 


= specific force vectors for the aircraft and 
simulator, respectively 


rotational velocity vectors for the aircraft 
and simulator, respectively 


perceived linear acceleration vectors for the 
aircraft and simulator, respectively 


perceived rotational velocity vectors for the 
aircraft and simulator, respectively 


DOWN. - perceived attitude vectors for the air- 





craft and simulator, respectively. 
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ЧЕТ, RESULTS 


A. EXPERIMENTAL RESULTS 
1. Subjective Responses 

Verbal responses recorded at the end of each task 
allowed the experimenter to understand such considerations 
as the learning curve, workload, etc., associated with the 
task. The act of commenting on each run gave subjects the 
opportunity to critically review the flight and to adjust 
their scores as opinions were formulated. This section is 
a brief summary of pilot comments pertaining to the 
experimental flights. 

Differentiation between the rotational and transla- 
tional motion cues was discussed with the subjects as the 
experiment progressed in hopes that more reliable responses 
would result. Following his fifth run, one subject attempted 
to further define his task: 

"I'm to separate pitch and roll from sideways, hori- 
zontal, or longitudinal axis motion as linear, whether 
it be throttle acceleration or just sideways gusts." 

Several pilots commented that they were more confi- 
dent of their responses during second and subsequent ses- 
sions as they familiarized themselves with the task and 
learned more about the characteristics of the motion to be 
Ea uo pilot mentioned that it was difficult to dis- 
cern whether motion cues were more realistic or his ability 


to control the simulator had improved. During a subject's 
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second session he commented: 


" 


| Tene oscillations Twas in the other day; at 
СЕБЕИ thought T was experiencing pitch oscillations. 
What they are is the linear, up and down, that I was 
reclined before; so I'm getting used to the motion.... 
at least understanding it better." 

All the pilots noted that the workload level varied 
significantly from run to run as the motion cues changed. 
Abnormal cues often caused subjects to react improperly; a 
response which presumably would not have occurred in a fixed- 
base simulator. One subject noted the difference between 
ene mocion and cockpit instrument indications: 

"I was getting a definite rotational pitch sensa- 
mromeana frequently corrected with [forward] stick.... 
ИШЕ I didn't want to do.” 

All subjects noted vibrations induced by the motion 
base mechanism. Normally turbulence masks these anomalies to 
some extent. Since it was not used in the experiment, the 
mechanical limitations tended to put a ceiling on the subjec- 
tive motion ratings. One subject was allowed a practice run 
with a minimal level of turbulence at the beginning of his 
second session. He commented: 

" . turbulence...simply detracts from the ability 
tomdurrerentiate...how the controls are functioning. 


I think I'm less able to determine how much of it is 
...turbulence or control response with the turbulence." 


MN correlation ot Ratings 
Scores from the three rating scales represent discrete 
data possessing ordinal properties. It can be argued that 
each rating scale attempts to define levels separated by equal 


intervals: it is not immediately evident that each subject 
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perceived the scales in this manner.  'fhus, it cannot be 
assumed that the data comes from an interval scale, an 
assumption for use of parametric statistics. As a measure 
of the reliability of the data, nonparametric correlations 
were calculated. Scores from the standard motion were ex- 
cluded from this and subsequent analysis since subjects were 
told which runs were the standard and instructed to use it 
as a comparative reference for the other 15 parameter sets. 
An average score was formed for each subject/motion/param- 
eter set combination analogous to each "cell" of the experi- 
mental design (see Fig. 2). Spearman rank correlation coef- 
ficients between subjects were computed for the linear, 
rotational, and combined ratings separately. The correla- 


tions with respective significance levels are shown in 


Male III. 
SUBJECTS 
1 2 3 4 3 

E 1.0 .5240* Aon .6810***  .1264 
B p 1.0 .1435 .5668** .1722 
E 3 1040 .2263 .5666** 
T 4 Lee „0250 
S 

5 Т0 


Gigmurtcance Levels: *p<.05, **p<.0l, ***p<.005 


Table III(a); Correlation Between Subjects for 
Linear Motion Ratings 
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SUBJECTS 


|. 2 3 4 5 

$ 1 10 148357* .1244 .6081 .4367 
U 
B 2 120 „2823 AS EE ‚4157 
5 
E D IEO >. 00; 3963 
€ 
T 4 120 ‚2485 
5 

5 1:79 

Table III(b): Correlation Between Subjects for 

Rotational Motion Ratings 
SUBES TS 
1 2 3 4 5 

S 1 10 "5449 . 5065: ЛОО кх „2466 
U 
B 2 E OUS pol s 24700" 
J 
Е 2 o .4481* .4968* 
С 
T 4 1.0 .2564 
S 

5 10 


mri Icanhce Levels: *0<.05, **p<.01, ***p<.005 


Table III(c): Correlation Between Subjects for 
Combined Motion Ratings 


Overall agreement between the five subjects was com- 


puted using the Kendall coefficient of concordance as shown 


in Table IV. 
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RATINGS 
Linear Rotational Combined 


.4483* 2149/95 * S943 


Significance Level: *p<.005 


EEENCSIV: Coefficient of Concordance Among Subjects 
för Motion Ratings 


The ability of subjects to repeat ratings given the 
same stimuli was investigated. Due to time constraints only 
41 subject/parameter set combinations were replicated. Cor- 
relations between the first and second observations for these 


combinations is shown in Table V. 


RATINGS 
Linear Rotational Combined 
.4882* 212359] .5559** 


Sismificance Levels: *p<.005, **p<.0005 


Table V: Spearman Rank Correlation Between Replications 


ge bvsris of Variance 


ml e15 of Variance techniques were used to deter- 
mine if different motion cues significantly affected the 
ratings and if interactions existed which would help explain 


the weak between-subject correlations obtained. Although 








the discrete rating scale data violates the assumptions of 
meerval data and normality of the error distribution, an 
approximate ANOVA was computed. Since the rating scale con- 
sists of essentially 15 levels separated by intervals de- 
signed to be equal, the assumption of interval data is not 
entirely untenable. Madill [Ref. 17] reports that the dis- 
tribution of the error term in the regression model for 
discrete rating scale data is often binomial in nature. 
Thus, the variance is not independent of the mean as re- 
quired for ANOVA analysis. “he transformation which attempts 
to equalize the error variance for the binomial distribution 


is the arcsin\V X transformation. A transformation of the 


Сот: 
| R-1 
R' = Jarcsin\ =a - 1.0 (1) 
where: R = raw rating scale score 


R' = transformed score 


s j sS The effect is minimal in the range 3.0 to 6.0 and 
tends to expand the ends of the scale. The ANOVA table is 
shown in Table VI. Due to the large percentage of missing 
data, the three-way interaction sum of squares is included 


in the error sum of squares with its degrees of freedom. 
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SOURCE dura ток Des: F RATIO 








Subjects (S) 4 83.91 20.98 44.24% 
Motion Cues (М) 2 10:3 . 54 226 

Parameter Sets (P) 14 119.99 goo 4.86* 
Sy M 8 13519 1.64 3.46* 
S x P 56 59271 1 76 5572: 
M x P 28 19225 247 .99 

error 235 211.91 ‚48 

Total 347 441.98 


Significance level: *р<.001 


Table VI: ANOVA Table for the Experiment 


A nonparametric analysis of variance was performed 
to support the results of the approximate ANOVA above. The 
subject/parameter set averages were tested using the Friedman 
two-way analysis of variance by ranks [Ref. 18]. The three 
motion cues were investigated separately. Results showed 
that both the linear and rotational motion ratings were sig- 
nificantly different at the .01 level. The combined ratings 
were different at the .001 significance level. 

A Duncan multiple range test with unequal replications 
[Refs. 19 and 20] was performed on the means of the parameter 
sets. Results using the .05 level of significance are shown 
in Figure 9. Any two means underscored by the same line are 


nol SIGNIfEICantly different. 
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2515 A BS = кү 
—r AAC AOS: 
25 4.0 de 5.0 O 6.0 
Figure 9: Parameter Set Averages 


Br SIMULATION RESULTS 
1. Fidelity Models 
The available signals at the simulation output con- 
sist of vectors estimating the pilot's perceived motion and 
attitude. Each human dynamic orientation program (see Fig. 8) 
computes perceived linear acceleration, angular velocity, and 
attitude vectors. Algorithms for comparing these time vary- 
ing signals were constructed based on the assumption that the 
greater the difference between corresponding vectors for the 
aircraft and simulator, the less the fidelity. The following 


vector variables are available as inputs to the fidelity 


function. (Notation is consistent with that used by Young in 
Ref. 12.) 

A . . . . 

Eu 2 so r Corvedq alroraft linear acceleration 

^ 

DM Mee ccclved simulator lincar acceleration 

Wa (6) = perceived aircraft angular velocity 

We (€) = perceived simulator angular velocity 

OS 

DOME E) = perceived aircraft attitude 


Su 





DOWN. (t) = perceived simulator attitude 


Miéxe: a = aircraft, s = simulator, © indicates a perceived 


quantity 
(E) 3meéHcates a time 


үп function 


Three different fidelity functions were constructed 
to be compared with the experimental data. The first is an 
average percent error model as suggested by Young: 


"When the [motion] sensations are clearly supra- 
threshold, the most likely candidate is just percent 
ase Ete ratio Of perceptual error to the correct 
Smee. When the model indications for 'correct' 
perceptions are subthreshold, it seems more logical 
to assess a large penalty for errors that are large 
compared to the threshold value." 


For each sample output (one per second) the following "cost 


» . . e 
to realism" indices were computed. 





IA A(t) 
AA AE SA (2a) 
max Ag (tJ, Ren] 
LA W (t)! 
Е = = С, c) (2b) 

пах( |9, (0), Wehr 

HS 
|A DOWN (t)| = C. (t) (2С) 

where: A = the difference between aircraft апа 
simulator vectors 
|x| = amplitude of vector x 


thr = threshold value 


Threshold values used in the simulation were those 


computed in Ormsby's vestibular modelling thesis [Ref. тр 
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^^ 
De = .005 "G's" 
^ 


еш 0146 raáadians/sec 


mae vector DOWN is a constant amplitude unit vector; thus, 
the cost index is simply the magnitude of the difference 
vector. Averages for each of the three cost indices were 
computed over each simulator run. Though the task was de- 
Signed to last three and one-half minutes, subjects often 
Weed over four minutes to complete it. Aircraft acceler- 
ation data from each run, up to four minutes, was used as 
input to the simulation. The following average cost values 


were computed: 


Ca = 1, mm (3a) 
С, = 1) c) (3b) 
S ibl 

D C5 ft.) Be) 


SEI 


С) 
lI 


where: n = number of output time frames 


Several subjects noted that during many runs there 
existed unexplained and unrealistic motion excursions about 
the rotational axis, but more pronounced in the translational 
cues. A measure using the same cost indices which is more 
sensitive to these rapid anomalies (if caused by the program- 
ming and not the hardware) is a root mean square (RMS) fig- 


ure. The second fidelity function computes RMS cost values 


as follows: 
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C. (4a) 

ON (4b) 

ad 2 

C. EC (ti) (4c) 
1=] 





It is not unreasonable to assume that pilot responses 


may simply be proportional to the amplitude of the error vec- 


Cor. 


The third fidelity function computes the average of 


each error vector; the numerators of equations 3a, 3b, and 


EC. 


Еу; 


n 


Б) азе (5а) 


1=1 
n 


| | |А®(=,)| (5b) 


ї=1 


>| 
ii 
I 


= | 
I 
IH 


n 


ES 
[A DOWN (t. )| (БС) 


i-1 


A total of 133 simulator runs were recorded success- 


Ol 
ii 
I = 


97 of which used wash-out parameter sets other than 


the standard set. Minimum, maximum, and average cost indices 


computed by the three fidelity functions are shown in Table 


ERI. 
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MINIMUM AVERAGE MAXIMUM 


С . 09959 1.0788 


> To 796 “7605 


© „0626 „0898 


en 2.0106 
b. | и 10406 71 


Cr : ° “T451 





Table VII: Cost Index Values Computed by the Simulation 


oeri mental Optimization 
Response of the simulation to changes in the wash-out 
parameters was investigated to determine its adaptability to 
experimental optimization methods such as an optimal gradient 
Search. If the simulation is to be an effective research 
tool for determining optimal control system variables, then 
YES response characteristics must be of a tractable form. 


Time did not allow for a full exploration of its response 
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"surface" by an optimization routine, nor were manual methods 
employed to search for a minimum "cost" solution. 
Linear and rotational wash-out filter gains were 


varied over the range of the parameters used in the experi- 


Meme. The response function was arbitrarily chosen to be: 
fZ E Z TE ~ ae 
(Ca C Cp) - C4 *€ + Cy (6) 


The linear gains, Ki, were fixed at different levels 
while the rotational gains, К, меге varied together from 
0.0 to 0.8 at intervals of .05. The response curves obtained 
are shown in Fig. 10. The basic parameter set used during 


these simulation runs was Pg* 


C. SIMULATION VALIDATION 

Validation of the simulation consisted of comparing the 
subjective fidelity ratings with predictions from the simu- 
lation. It was then a question of which ratings and cost 
indices to analyze. Since the strongest correlations be- 
tween subjects were for ratings of the combined motion cues, 
the combined ratings were compared to the following unweight- 


Semlinear combinations of the cost indices: 


SAS + Cp (7a) 
u Ре c 

ma C - Cp (75) 
A +W +D (7c) 


Spearman rank correlation coefficients were computed for 


each subject and cost index combination. (Fidelity ratings 
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Rotational Gains, K, 


Simulation Response to Varying Filter Gains 
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were subtracted from a constant to yield scores propor- 


tronal to the cost to fidelity.) The results are shown in 


mole VIII. 


— — — — — ہے‎ 2 ar NO. 
Ca * C, * Cp Ca + C. + Cp, A DI RUNS 


.4147* NES 220,3 Кк 
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s ance levels: *p<0.1, **p<.05, @**p<.025 


Table VIII: Spearman Rank Correlations Between Subjects 
and Simulation Cost Indices 


Weighted sums of the cost indices were computed by mul- 
tiple linear regression techniques in order to improve the 
correlations. Using the assumptions for parametric methods 
given for the analysis of variance above, a regression model 


юге һе form: 


: — od rs 9 
К = аус + ась + asCp + a, (8) 
Mere: RK. =8 = к, 
ЕЕ > combined motion rating 


regression coefficients 


£ 
II 
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was used to compute regression coefficients for each subject 


and for all subjects combined. Results are shown in Table IX. 


Regression Coefficients 
Sig.@ 
MULT. R Level 


257-02 92 .4926 ‚1826 
эе] че энн [эзе эе [шш 


8. == 9-096 4.999 „4589 +2030 
“IDO 94530 2o DU 26053 ‚2442 „1245 


@ Significance level associated with the ANOVA F ratio for the 
regression sum of squares 












(n +3 O tJ G, шс O0 





д 
Subjects 





Siente cance levels: *p<0.1, **p<0.05 


Table IX: Regression Coefficients for Combined Ratings 
versus RMS Cost Indices 
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TU DISEUSSION 


Results of the experiment tend to reject the hypothesis 
that similar motion cues were presented to the subjects. As 
shown in Figure 9, a cluster of nine parameter sets exists 
toward the center of the range of means. Parameter sets one 
arce appear significantly better than the others, while 
E5 9, IT, 14, and 15 are significantly inferior to the 
others. The fact that the center group of parameter sets 
produced indistinguishable motion cues is a probable explana- 
tion for the weak correlations between subjects. Parametric 
Pearson product-moment correlations were computed yielding 
similar results at comparable significance levels. The 
highly significant coefficients of concordance also indicate 
that ordinal agreement between subjects was less than antic- 
ipated. The overall higher correlations for the combined 
motion ratings infer that subjects were capable of yielding 
more reliable scores after rating both linear and rotational 
motion cues on an absolute scale, followed by a pairwise 
comparison with the standard motion. The higher correlations 
could also be explained by the fact that several subjects 
said their responses resulted from an overall impression of 
the motion cues rather than a subjective average of the 
linear, rotational, and comparative scores. Two subjects 
rated the combined motion cues higher than either linear or 


rotational scores for several runs. 
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Comments by the pilots indicate that a large percentage 
of the variance of the pilot ratings is likely due to un- 
realistic dynamics of the motion drive mechanism. High 
frequency vibrations in the system were not as distracting 
and did not seem to lessen fidelity as much as the low fre- 
quency, larger amplitude anomalies experienced by the sub- 
jects. Whether these were caused by the software or 
hardware ls not known. 

Cost indices computed by the simulation show large dif- 
ferences between outputs of the vestibular models. Errors 
ranged from 58 to 148 percent of the perceived aircraft 
motion vectors as shown in Table VII. Whether this typifies 
the capabilities of the S.01 simulator or is largely a result 
of programming limitations and inaccuracies is not readily 
apparent. А significant amount of error is most likely due 
to roundoff, precision limits, and the discrete nature of the 
program. Motion histories of the aircraft and simulator have 
cycle times of .05 seconds, while the output rate of each 
vestibular model is one per second. This represents a con- 
encdereblerjeoss of information. If the output sampling rate 
equaled the input rate, it may have been more responsive to 
the rapid motion anomalies noted by the subjects. A quick 
excursion lasting less than a second may not have been re- 
flected in the output of the human dynamic orientation model. 

The aircraft model used during the experiment was that of 
the F-15 fighter. Ideally, subjects with recent F-15 experi- 


ence should have been chosen, but due to time and availability 





constraints selection of the pilots was less than optimal. 
K (r IC that this factor contributed greatly to the lack 
GE aigher correlations of the ratings between subjects. 

This conclusion was supported by the weak correlations be- 
eween the first and second replications of the parameter 
ers also implies an inability of the subjects to 
repeat the task as well as inconsistent responses between 
subjects. 

Response of the simulation to changes in wash-out filter 
gains indicated a definite tendency for convexity in the 
neighborhood of minimum output. This can be explained in- 
tuitively. When the gains are relatively low, the lack of 
motion cues produces a large discrepancy between perceived 
aircraft and perceived simulator motions. When the gains 
are high, cues become overly rapid, exagerated, and fre- 
quently cause the simulator to reach its "soft" programming 
limits. When reached, the limits force the linear and ro- 
tational cues to become uncoordinated, reducing fidelity. 
The simulation output, defined by equation 6, varied over a 
small range as the rotational gains were varied as shown in 
ure 10. For each level of the linear gains, the output 
Varied only 2 to 4 percent. Parameter sets which produced 
significantly better motion cues, Ру and Pz, were comprised 
of filter gains set at 0.3. The tendency for the simulation 
output to minimize at approximately 0.3 indicates a capacity 
to predict subjective preferences for motion cues produced 


by these parameter sets. The simulation was also run with 
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the linear gain, Ку, set at 0. Response was less than that 


for K, set to 0.1 throughout the range of K... This implies 


that optimal motion cues include very little linear acceler- 


ation for the specific flight used as input to the simulation. 


Comparison between the simulation output and subject 


responses is summarized as follows: 


Positive correlations were obtained for the first two 
fidelity models, equations 7a and 7b (see Table VIII). 
The third function, equation 7c, showed little correla- 
tion with the exception of subject five. 

The second fidelity function consistently produced 
higher and more significant correlations infering that 
the RMS cost indices are a better method for simulating 
subject responses. 

Linear regression using the RMS cost indices as inde- 
pendent variables and the combined motion cue ratings 
as dependent variables (equation 8) showed little con- 
sistency of regression coefficients among the five 
subjects (see Table IX). 

Linear regression analysis yielded multiple correla- 
tions not appreciably better than the ccrrelations 

for the unweighted cost index combination. 

Very little correlation was found between ratings of 
me lingar and rotational motion cues and the corres- 
ponding cost indices, such as the rotational ratings 


paí 
ала Cur 
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Nine of the 15 wash-out filter parameter sets were 
grouped in the center of the range of mean responses as 
shown in Figure 9. Since these did not represent signifi- 
cantly different means, Spearman rank correlations were com- 
puted between the subjects' combined scores and the second 
Fidelity function, equation 7b, for the remaining six param- 


eter sets. The results are shown in Table x. 


SUBJECTS 
S 
1 “2 5з 54 "а 
Correlation „9710** . 0748 .6545* .5000 .7660* 
No. of Runs 6 7 10 3 8 


sion asmuesance devels: *p«.05, **p« .01 


Table X: Spearman Rank Correlations for the Remaining 
Six Parameter Sets 


With the exception of subject two, the subjects showed a 
marked increase in correlation with the simulation results. 
Although the sample sizes for this post hoc analysis were 
small, three of the five correlations were significant. It 
was noted that the correlations compared closely, rankwise, 
to the flight experience each pilot had in high performance 


ПС Tera aS indicated in Table I. 
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VI. RECOMMENDATIONS 


Recommendations for further research include suggestions 
for improvement of the experimental data base and the com- 
puter simulation. Better reliability and consistency in 
subject responses and a more accurate and responsive simu- 
lation would improve the prediction capability of the 
Simulation. 

Further studies should employ subjects proficient in the 
Simulated aircraft. Since the results appeared to be strong- 
ly affected by the subjects' ability to repeat the task, use 
of experienced test pilots familiar with flight dynamics 
would help create a more reliable data base. Future re- 
search should include preliminary studies to determine other 
control parameters which when varied would create motion 
cues more significantly different. Discrete rating scale 
aris difficult to analyze with classic analysis of vari- 
ance techniques, therefore, methods for obtaining pilot re- 
sponses from a continuous, interval scale should be pursued. 

Time did not allow for elimination of the inefficiencies 
of the simulation.  Recomputation of the human dynamic orien- 
tation program's input and output model parameters would most 
likely improve the responsiveness of the model. Converting 
the program to double precision would reduce the inaccuracies, 
but double the computer memory requirements. The core re- 
quired as listed in Appendix C and as executed on the IBM 


360/67 Operating System was 458K. The entire flight 
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acceleration data set was read in at once. Other time his- 
tories, such as the simulation motion vectors, were computed 
creating large core requirements. If each time frame were 
more ccedi through the simulation before the next was read in, 
the core requirements would be reduced by 75 percent. 

It was assumed that an accurate facsimile of the simu- 
lator motion was generated by the drive program. Since the 
lead compensation network was bypassed in the simulation, 
follow-on studies should compare the output of the drive 
program with actual simulator motion histories. This is a 
possible source of considerable error. 

The motion vectors for the simulator and aircraft could 
have been compared directly without use of the human dynamic 
orientation program. A similar analysis of the difference 
between these vectors would provide a way of validating 
Ormby's model. If the method incorporating the model cor- 
related appreciably higher with subject ratings, a reasonable 


validation of the human dynamic orientation would be achieved. 
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APPENDIX A 
TNSTRUCTIONSS TO SUBJECTS 


A. PURPOSE OF THE EXPERIMENTS 

The intent of this study is to analyze pilot response to 
varying degrees of simulator realism or fidelity. Subjective 
ratings by commercial pilots like yourself will help con- 
struct a mathematical model of the human process of compar- 
ing simulator motion characteristics with those of the 
Simulated aircraft. Ultimately, our goals are to validate a 
model of human dynamic orientation and to optimize the drive 


programs of aircraft simulators. 


Br TASKS 

You will be asked to repeat two familiar tasks, each 
image trem three to three and one half minutes. The first 
will be a basic airwork pattern, designed to employ the 
maneuvers common to operations in the terminal phase of 
flight. The second task is a full ILS approach to a wave- 
off. The basic airwork task will be repeated approximately 
fifty times, while the drive program parameters are varied 
to produce different degrees of fidelity. At the end of 
each task you will be asked to rate the characteristics of 
the motion using three rating scales discussed below, as 
well as provide your subjective opinion of the motion, task, 
etc. During subsequent sessions, you will rate the motion 
following the ILS approach task, repeated roughly the same 


number of times. 
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Each session will consist of approximately 12-15 flights 
and last no longer than 80 minutes. Detailed diagrams of 
each task will be available during the runs. Additional 
details will be provided before the sessions begin. 

mic simulated aircraft in this study is the F-15, a high 
performance, fighter aircraft. Although you probably have 
had no experience in the F-15 itself, your background in 
high performance type aircraft was a major consideration in 


your being selected as a pilot for this study. 


SUSE OF THE RATING SCALES = OBJECTIVE RESPONSES 

Fach trial run will employ a different set of drive pro- 
gram parameters. A standard set of parameters will produce 
a "Standard motion" which will be presented during the first 
run and every fourth run thereafter. Repetition of the 
standard motion will reinforce your perception of it, allow- 
ing you to make comparative evaluations of the trial runs. 
A typical sequence of 12 runs per session will be presented 


as follows: 


Trial No. 2 3 4 6 7 8 E 1010 Ji 172 


where indicates that the fifth run of the session will 
be the "standard motion." 


1. Component Rating Scale 





Evaluate the linear (translational) and rotational 
components of the motion following each simulator run using 


the component rating scale. Rate the motion characteristics 
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based on the descriptive phrases provided at each level of 
the scale. These subjective descriptions should be used 
merely as an aid in your evaluations. Responses should 
reflect consideration of: 

ST RESPONSE EO CONETEOL npn e Sumus o 
reaction to movement or pressure on flight controls. 

b. Motion Deficiencies - motion deviations from 
experienced or anticipated aircraft motion, not a result of 
Gentrol inputs (e.g. response to turbulence, oscillations, 
SEC.) 

c. Other - any other discrepancies in the motion 
dynamics which you detect and consider significant. 

Your responses should be formulated in reference to 
the absolute scales as well as in comparison to your rating 
of the standard motion. 

22 Comparative Rating Scale 

Rate the trial runs in comparison the the standard 
motion using the comparative rating scale. Brief descrip- 
tions of each level of the scale should aid your evaluation. 

5, Combined Rating Scale 

Using the descriptive phrases on the component rating 
scale, rate the trial run on an overall or combined basis. 
Reference should be made to your scores given on both the 
component and comparative rating scales. 

4. General 
a. If you cannot choose between two adjacent levels 


on either of the scales, a "half" level score is allowed, 
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provided careful consideration is given to both adjacent 
levels. 

b. Ensure that your rating of the "standard motion’ 
1s recorded on the scale and referred to while rating trial 
runs. 

c. Basic performance measures will be recorded 
only during the ILS approaches. Although your performance 
will be compared to that of other pilots, our interest lies 
in how your performance varies with changes in fidelity. 

d. Try to ignore changes in your performance on 
the task as each session progresses. 

e. Your evaluations should reflect your assessment 


of all the motion cues detected. 


BS. SUBJECTIVE COMMENTS 

Pie rsracıme each trial run, brief cemments on the rtems 
listed below are encouraged. It is not necessary to remark 
on each issue, but only on those you feel significant or 
pasercularly relevent to the trial run. 

l. Your ability to rate, separately, the linear and 
rotational components of the motion. 

2. Subjective description of the motion characteristics. 

3. Difficulties encountered with the task as it is 
affected by the particular motion characteristics. 

4. Impressions of the evaluation task. 

5. Duration of the task versus your ability to evaluate 


the motion. 
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6. Physical discomfort (fatigue, nausea, etc.). 
7. Workload level in terms of the attention level 


required to perform the task. 
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APPENDIX P 


WASH-OUT FILTER PARAMETER SETS 


Each wash-out filter (one for each degree of freedom) 
is described by three parameters. The values of the param- 
eters used in the experiment are shown in Table Bl. Where 
only one figure is shown, that value was used for all three 
channels. Natural frequencies for the translational chan- 
nels are in the order of roll, pitch, and yaw. The 


parameters are defined as follows: 


К, - Linear gains for the translational channels 
K, - Linear gains for the rotational channels 
W, T Natural frequencies for the translational channels 


- Natural frequencies for the rotational channels 
- Damping factors for the translational channels 


a т ШО ШЫ Пс factors for, the rotational cehannels 


The "standard motion" parameter set is denoted by Pg- 
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Parameter 
Set No. 


Table Bl: 
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APPENDIX © 
SIMULATION SOURCE PROGRAM LISTING 


The simulation, written in FORTRAN IV H, consists of sub- 
routines which perform the functions shown in the block dia- 
сам in Figure 9. The program listing in this appendix is in 
the configuration used to perform the experimental optimiza- 
tion described in section III.B.2. The main subroutine 
(called SIM) controls the other subroutines and prints the 
wash-out parameter set and messages displaying the progress 
of the program execution. A main program (not listed) reads 
її the values for the wash-out parameters, varies them, and 
iteratively calls the main subroutine. The flight acceler- 
ation time histories are read in by subroutines READ. The 
model parameters used by the human dynamic orientation pro- 
gram (subroutine HDO) are those obtained with the program 
listing contained in Ref. 21. The simulation was executed 
on an IBM 360/67 Operating System using single precision, 
floating point variables. 

Subroutines MCPY and MINV are IBM system library sub- 
routines called from subroutine COTRN to perform matrix 
operations. MCPY copies one array variable into another. 


MINV inverts a square matrix. 
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